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Research (WCER) , and project managers from the Council of Chief State School 
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Executive Summary 



Last year, schools and teachers in 1 1 states participated in a study of the “enacted curriculum” in 
mathematics and science classrooms. Over 600 teachers across the states completed self-report surveys 
that covered the subject content they taught and the instructional practices they used in their classes. 
The goals of the study were to: 

• Measure differences in instructional practices and curriculum content among 
teachers and schools, 

• Determine if there are differences in math and science teaching that are related to 
state policy initiatives and state standards, and 

• Demonstrate the use of “surveys of enacted curriculum” to analyze classroom 
practices and to produce useful analyses and reports for educators. 

The study of enacted curriculum was a collaborative effort involving state education leaders in science 
and mathematics education, researchers from the Wisconsin Center for Education Research (WCER), 
and project managers from the Council of Chief State School Officers (CCSSO). Educators and 
researchers worked together to develop survey instruments that would gather reliable data from teachers 
and students, as well as formats for reporting survey results that would communicate key findings to 
educators. The project received core support from the National Science Foundation (NSF). 

This summary report provides an overview of some of the findings from the study and examples of how 
the data on enacted curriculum can be reported and used. It is intended to help educators and decision- 
makers identify the kinds of information that would be available from the “surveys of enacted 
curriculum” and to suggest ideas and strategies for more expanded use of these kinds of surveys and 
data by educators at the school, district, and state levels. 

Purposes for a Survey of Enacted Curriculum 

The survey approach used in this study offers a practical research tool for collecting consistent data on 
mathematics and science teaching practices and curriculum based on teacher reports on what is taught 
in classrooms. The enacted curriculum data give states, districts, or schools an objective method of 
analyzing current classroom practices in relation to content standards and the goals of systemic 
initiatives. The methods of aggregating and reporting survey data allow educators to analyze differences 
in classroom practices and curriculum among teachers and schools with varying characteristics, and 
districts and states can analyze differences in classroom curriculum related to state initiatives or to the 
state or district standards in math and science. 

The data are collected by teacher self-report with surveys designed for elementary, middle, or high 
school teachers. Teachers are asked to report on the range of practices and subject areas covered during 
the course of the school year and to provide information on the school, class, and their preparation and 
professional development for teaching. 
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Main Topics and Issues Reported 

The major concepts underlying the Survey design are from state content standards, state initiatives in 
science and mathematics education, and prior research studies on classroom instructional practices and 
curriculum content. The Survey is intended to answer many of the key questions educators and 
policy-makers have about patterns of classroom curriculum and practice across classrooms, schools, 
districts, and states. The following outline of major concepts in our framework and sample summary 
findings from our 1999 study typify the kinds of issues and questions that can be explored with the 
Survey data. 

1 . Active Learning in Science 

Question: To what extent are students involved in active, hands-on learning approaches 
in science class? 

• Sample survey data suggest one-fourth of science class time is spent on hands-on 
science or laboratory activities, but there is wide variation among schools. 

• Survey data allow comparison of active science methods in schools that are involved 
in state initiatives and science teaching in typical schools. 

2. Problem Solving in Mathematics 

Question: To what extent are students in math class learning problem-solving and 
reasoning skills, and learning how to apply knowledge to novel problems? 

• A majority of teachers report teaching problem solving in math, but teachers use a 
wide variety of instructional practices, such as small groups, writing, data analysis, 
and applying concepts to real world problems. 

• Differences are found in the types and depth of instruction of problem-solving activities 

between initiative and comparison schools. 

3. Mathematics and Science Content in Classrooms 

Question: How does math and science content taught in classes compare to the goals 
outlined in state and national standards? 

• In middle grades math and science, most recommended standards are covered, but the 
level of expectation and depth of coverage vary widely among schools and classes. 

• Data reveal differences in extent of teaching science content across the standards and 
the extent of articulation between grades. 

• Schools differ in extent of emphasis on algebra, geometry, and data/statistics at 
elementary and middle grades. 

4. Multiple Assessment Strategies in Math and Science 

Question: What methods of student assessment are used in class, and are the strategies 
consistent with goals of learning in content standards? 

• A majority of teachers use multiple assessment methods in math and science classes, 
but infrequently use extended student responses that require student explanation and 
justification of answers. 

• In science, the survey data allow analysis of differences in the use of performance 
tasks, or hands-on activities, for assessment in class. 
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5 . Use of Education Technology and Equipment 

Question: How is education technology, e.g., calculators and computers, used in math 

and science instruction? Do teachers have science equipment available in their classes, 

and how often is it used? 

• A majority of elementary and middle grades teachers use calculators in teaching math; 
graphing calculators are available in the typical grade 8 classroom but are rarely used. 

• The average elementary classroom has basic science equipment, but rate of use varies 
widely among teachers. 

6. Influences on Curriculum and Practices 

Question: What effect do state and national standards for science and math learning have 

on the curriculum taught in classrooms? 

• State frameworks/standards and national standards are reported by most teachers as 
strong positive influences on their curriculum. 

• Survey data allow comparisons of degree of influence on curriculum of state and 
national standards, textbooks, state and district tests, and teacher preparation and 
knowledge. 

7 . Alignment of Content Taught with State Assessments 

Question: Do state assessments reflect what is being taught in classes? 

• Analysis of teacher reports and state assessment items show that tests cover a 
narrower range of expectations for students than reported instruction, with tests 
focusing more on memorization, facts, and performing procedures and less on solving 
novel problems and applying skills and concepts. 

• The data on alignment between teacher reports on instruction/content and state 
assessments allow teachers and assessment staff to examine the areas of weakness and 
strength of tests and classroom practices. 

8. Teacher Preparation 

Question: How well prepared are our teachers to teach science and mathematics? 

• The survey data show how well prepared teachers are for using innovative teaching 
strategies and handling students with varied needs and capacities. 

• Middle grades teachers in math and in science receive more professional development 
than elementary teachers both in methods of teaching and subject content. Teachers 
report very positive reactions to professional development related to standards, 
curriculum, and assessment in their recent activities. 
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Study Objectives and Design 



The design for the Survey of Enacted Curriculum in mathematics and science conducted in 1 999 came 
from earlier research and development projects of the Council of Chief State School Officers (CCSSO) 
and the Wisconsin Center for Education Research (WCER). CCSSO had worked with state education 
leaders in developing content standards and assessments in science and mathematics. WCER 
researchers had tested the validity and usefulness of a survey approach to collecting reliable, 
comparable data on classroom curriculum and practices. 

The movement of states toward standards-based reform in mathematics and science produced strong 
interest in reliable data for evaluating the effects of reforms. CCSSO and WCER recognized the 
possibility of applying research-based models and instruments for studying curriculum to broader 
purposes of reporting indicators of curriculum and instruction that could be used by policymakers and 
educators. CCSSO submitted a proposal to the National Science Foundation (NSF) to lead a study 
of change in curriculum and instruction related to state standards and state initiatives for improvement 
of mathematics and science. 

State Participation. States were asked to voluntarily participate in the study if they were 
interested in gaining information on effects of their reform efforts and gaining knowledge about the 
development and use of a survey approach to analyzing curriculum. In 1998, 1 1 states chose to 
participate, and state specialists in mathematics, science, assessment or evaluation were invited to join 
the study management team. The states chose a sample of 20 schools at each of two grade levels (e.g., 
elementary, middle) for the study. Half the schools selected had high involvement in their state’s 
initiative for improving math or science education (“Initiative” schools), and the other half were 
schools with less involvement but were similar to the first group based on student demographics 
(“Comparison” schools). 

Data Collection. The Survey of Enacted Curriculum is primarily conducted through teacher 
self-reports of classroom instruction over the course of a school year. Two teachers per grade level 
and subject were selected by the principal of each school. Basic information was collected about the 
schools, and a student survey was conducted in one-fourth of the classes for data validation. Ten of 
the 1 1 states chose to focus their school selection at the elementary and middle school level, and one 
at the middle and high school level. 

The Survey for a specific grade level and subject includes more than 1 50 questions covering: 

• Instructional Practices, including classroom activities, assessment, influences on 
curriculum, and use of technology and equipment; 

• Subject Content, including curriculum topics taught by expectations for learning; and 

• Teacher Characteristics, including teacher education, professional development, and 
teacher reports on school conditions. 

Teachers completed the survey individually, and many used their own time outside of school. 
Teachers were guaranteed confidentiality, and the main incentive was to contribute to their state's 
study of reform initiatives in math and science education. At the same time, they were assured data 
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would not be used for school accountability or teacher evaluation purposes. In Spring 1 999, CCSSO 
obtained completed Surveys from a total of 604 teachers across the 1 1 states. 



Examples of Data Reporting 

Selection of schools and teachers for the study in each of the 1 1 participating states was based on the 
degree of school involvement in the state math or science reform initiative. The collected data from 
the sample schools present sufficient numbers of responses to provide meaningful statistics, such as 
mean and standard deviation, and the numbers allow analysis of the significance of reported differences 
related to curriculum and instructional practices in “Initiative” vs. “Comparison” schools. The results 
from the 1 999 Survey reported in the following charts are not nationally representative, nor are they 
representative of all mathematics and science teaching in schools in the 1 1 states. 

The data presented in this report are primarily intended to demonstrate to educators and policymakers 
the kinds of analyses and reporting that are possible with the Survey of Enacted Curriculum. That is, 
the kinds of summary data and charts displayed in this report can be produced for an educational 
system should a state, district, or school conduct the Survey with all their teachers or with a randomly 
selected, representative sample of teachers. The kinds of results reported for the 1 999 sample of 
teachers and schools in this summary report effectively illustrate the potential of the how the Survey 
of Enacted Curriculum could be used for future applications. 

Selected Data Charts. Keeping in mind the uses of our study and the limitations of the data, 
we present sample data charts from the 1999 Survey focusing on eight concepts that are central for 
analyzing differences in classroom curriculum and for studying effects of state initiatives and standards: 

• Active Learning in Science 

• Problem Solving and Reasoning in Mathematics 

• Mathematics and Science Content Being Taught 

• Multiple Assessment Strategies 

• Uses of Educational Technology and Lab Equipment 

• Influences on Curriculum and Practices 

• Alignment of Content Taught with State Assessments 

• Teacher Preparation 

These sample charts include only a portion of the data from the Survey of Enacted Curriculum and 
represent only a sample of what can be done with the results. We hope these sample results stimulate 
the interest and curiosity of educators to either look further at the 1 999 Survey results or to conduct 
their own Survey using this model. Data from the 1999 study are available by state and aggregation 
across the 1 1 states. Educators who want to try their own application and use of the Survey tools in 
their state, district, or school can obtain them by contacting CCSSO. 

Reporting Format. The Survey results are reported and analyzed using several formats: Item 

Profiles, Summary Scales, and Content Maps and Graphs. 
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Item Profiles present data from individual survey questions, grouped by topic and item 
format (see middle of Chart 1). The data are shown in horizontal bar graphs. The mean 
is indicated by a solid vertical line, and the shaded bar represents responses that are one 
standard deviation above the mean and one standard deviation below the mean. 
Generally the responses at the mean and within the bar represent about two-thirds of all 
responses to a question. The number of teacher responses per group (e.g., middle, 
elementary) is reported in parentheses, (e.g., 104). 

Summary Scale is an average score for a group of 5 to 8 questions in the survey 
centered on a specific concept underlying curriculum or instruction, e.g., active learning 
in science (see top of Chart 1). Scales are formed by purposeful selection of items, and 
statistical analysis of responses to determine scale reliability (e.g., .78 for scale in Chart 
1). The selected scale items typically cut across different sections of the survey, and 
items may have different kinds of responses. The scale measures are “standardized 
scores,” meaning the average score for the scale for the whole group of teachers is set 
equal to 0, and the standard deviation (a measure of variation in responses) for the whole 
group is 1. Scale score differences would mean that sub-groups of teachers, e.g., 
elementary vs. middle school teachers, differ on the concept being measured. 

Content Maps and Graphs. Teachers report time spent on subject content during the 
year using a content matrix covering topics and expectations for learning. Responses 
of teachers are aggregated by grade level and reported with two statistical software 
programs: a mapping program which gives a three-dimensional picture of variation in 
time across the whole curriculum (see Chart 3), and histograms, which show average 
percent time by topic and expectation (see Chart 4). 

The Analysis Guide (following Examples) provides Survey sample items and a section of the content 
matrix as well as further explanation of how statistics are generated from survey data and reported in 
these formats. 




Using Data on Enacted Curriculum in Mathematics and Science 




Active Learning in Science 

Chart 1 displays data from the science teacher survey results across the 1 1 participating states in the 
1999 Survey of Enacted Curriculum. Data related to Active Learning in Science are reported in two 
ways: a Summary Scale that combines the data from eight survey items, and individual Item Profile 
graphs for seven items. The Scale provides a reliable summary measure of differences across the 
teacher sample in Active Learning and the degree of variation among teachers. We display individual 
item results to allow readers to see the relative amounts of class time spent on classroom practices 
related to active science learning. 

Active Learning is one of the central concepts in state and national standards for student learning in 
science that underlies the Survey and the study of state initiatives. The data in Chart 1 are intended to 
illustrate how the Survey of Enacted Curriculum can be used to analyze the extent of active science 
learning, and the kinds of classroom practices being used. We can also analyze differences in active 
learning strategies across the whole sample and differences between Initiative and Comparison schools. 
Several examples of Survey results are demonstrated: 

• Percent of Time. The first set of item profiles, in the middle of the chart, show the percent 
of science class time that are reported spent on three kinds of active learning: a) collecting 
information, b) doing lab activities or investigations, and c) students working in groups. 
Teachers reported in our 1999 Survey that from 10% to 15 % of time in science class was 
spent on each of these activities. In the left column we see there is wide variation among 
classes in use of active learning practices at both elementary and middle grades— varying from 
5% of time to over 1 8%. This amount of variation is equivalent to each type of activity 
varying from once per month to once per week. 

• Summary Scale. Both the summary scale on active learning, at the top, and the item profiles 
indicate that elementary classes spend more time on active learning in science than middle 
grades classes. The second and third columns on the scale results indicate that science 
classes in Initiative schools spent significantly more time on lab activities and investigations 
than classes in Comparison schools. 

• Experiments/Investigations . In the bottom part of Chart 1, more detailed information is 
presented on what students do during experiments or investigations. The Survey included 
eight items— with four results highlighted in the chart. In a typical classroom experiment or 
investigation, students would be expected to be engaged in a number of these activities in 
combination or sequence. Thus, it is useful to examine groups of activities with similar time. 
One can note that three activities are conducted more often: "use science equipment," 
"follow step-by-step directions," and "make tables, graphs or charts." Students spend less 
time "changing something in an experiment to see what happens." One can also note in 
column two and three that the elementary and middle grades classes in Initiative schools had 
significantly greater use of science equipment/tools in experiments or investigations. 

• Further Survey Data. In the Appendix, we report results on "Use of class time during the 
most recent science instructional unit," where teachers allocated time by major activities. 
For example, the average teacher reported one-quarter of science class time was used for 
hands-on activities, investigations or experiments. Teachers in Initiative schools reported 
significantly more time on active learning in science than those in Comparison schools. 
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Reasoning and Problem Solving in Mathematics 

Both reasoning and problem solving were curriculum standards for K- 1 2 math education set out in the 
NCTM Mathematics Standards (1989), and this learning goal is found in the state standards for the 
states involved with the study (Blank, et al., 1997). Key questions related to this standard are: "How 
much instructional time do teachers spend on this goal?" "What do teachers mean when they use the 
term 'problem solving'?" The results in Chart 2 illustrate how Survey data can be used to answer such 
questions. 

Summary Scale. The scale on reasoning and problem solving, displayed at the top of the Chart, is 
comprised of seven Survey items that, taken together, provide a reliable index of the degree to which 
teachers are focusing math instruction on improving students’ reasoning and problem solving skills and 
knowledge. (The average teacher scale score is set at 0 using a statistical procedure.) 

• The scale results show there is wide variation in time focused on mathematical 
reasoning and problem-solving activities. 

• There is little difference in overall results for middle and elementary math classes, but 
there are significantly higher scores for Initiative elementary math classes than for 
Comparison classes. This finding indicates that elementary mathematics teachers in 
schools participating in one of the state reform initiatives spent significantly more time 
on reasoning and problem solving activities. 

Item Profiles. When interpreting the results from questions on mathematical reasoning and problem 
solving, it is important to know how teachers interpret the term “problem solving.” For example, do 
they mean completing exercises in a text or solving novel problems (those for which students have no 
ready procedure)? The second section of the chart identifies these distinctions. For example: 

• An average of 30% of time in elementary and middle grades math in “problem solving 
activities” is spent on computational exercises from texts or worksheets. Over one- 
fourth of time in middle school math is spent on word problems from texts or 
worksheets. There is wide variation in time on these activities— from under 20% to 
over 50%. 

• Middle grades classes spend significantly more time on applying math concepts to 
real-world problems, while elementary classes spend significantly more time on 
making estimates or predictions. 

• The math classes in Initiative schools spend less time on computation exercises or 
word problems from texts than classes in Comparison schools (about 10% less time 
for each). In Initiative schools, there is significantly more time spent on writing 
explanations at elementary and middle grades. Significantly more time is spent in 
Initiative schools on solving novel problems in middle grades math. 
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Chart 2 

Reasoning and Problem Solving in Mathematics 
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Mathematics and Science Content Being Taught 

The Survey of Enacted Curriculum incorporates an innovative approach to data collection on subject 
content taught in class. Teachers are asked to report the amount of time spent on content in 
mathematics or science using a “subject content matrix.” The two dimensions of the matrix are 
content topics and expectations for students. Teachers first identify the major math or science 
standards covered in class, then amount of time spent teaching specific topics, and then report the 
expectations they had for student learning (e.g., memorize, use procedures, analyze information). 
Teachers report subject content taught over one school year. (See the Reference Guide section for 
explanation of content matrix.) 

The main advantages of content matrix design for data collection on content taught is its consistency 
with standards-based learning, as described in national professional standards (NCTM, 1989; AAAS, 
1993; NRC, 1995) and most state standards. In our approach, curriculum is viewed as a combination 
of the math or science disciplinary knowledge to be learned (e.g., geometry) and the skills and 
capacities that students are expected to gain through instruction (e.g., solve real-world problems). 
Content responses from teachers were aggregated across the 1 1 states by teacher grade level 
(elementary, middle) and by Initiative vs. Comparison schools. 

Science Content 

Chart 3 presents a Cross-State Sample "content map"— a three-dimensional map of topics by 
expectations by time-that reflects the 1999 Survey results from elementary science teachers. Chart 
4 presents a Cross-State Sample "content graph" that reflects the same data. Most of the reporting 
elementary teachers were at grade 4 with some at grade 3 and grade 5. 

• Life Science, Physical Science, and Earth Science were reported by teachers as taught 
from 20 to 30% of time on average. Measurement and Calculation in science 
averaged 12% of time. The aggregate category "Nature of Science" averaged nearly 
20% of time— this category includes teaching scientific method, history of science, 
science and technology, and science-health-environment. The most striking aspect 
of data on science topics is the wide variation in time spent in each category, 
especially among Initiative schools. 

• All six expectations for science learning were reported an average of 15% of time, 
except Understand Concepts which averaged over 20%. In science, Memorize means 
learning facts, definitions, terms, and formulas. Understand Concepts means student 
explanations, observe and explain teacher demonstrations, explain methods of science, 
and display data— with time varying from 10% to 35% of time among Comparison 
schools. 

• Teachers in Initiative schools reported slightly more time spent on Nature of Science 
than teachers in Comparison schools. Initiative classes had higher expectations for 
Analyzing Information about Nature of Science and Understanding Concepts, and 
slightly higher expectations for Conducting Experiments than did Comparison classes. 



Using Data on Enacted Curriculum in Mathematics and Science 

12 fn 



Mathematics Content 

Chart 5 presents a Cross-State Sample “content map” for middle grades mathematics; Chart 6 presents 
a Cross-State Sample “content graph” for the same data. Across the states, all the middle level math 
teachers were teaching grade 8 and reported on grade 8 math teaching. The content maps reflect 
patterns of responses from teachers in Initiative and Comparison schools and identify the intersection 
of content topic by expectation for student learning. In the content graphs, each cell has bars 
representing average percent time for Comparison vs. Initiative teachers, and the row and column 
marginals indicate the mean and range of responses for each topic area (e.g., number sense) and type 
of expectation (e.g., memorize). 

• Middle level (grade 8) math teachers reported that the most time was spent on 
Number Sense (average 25% time) and Algebraic Concepts (average 30% time). 
Geometric Concepts was reported taught less (average 15%), Measurement and Data 
Analysis/Statistics were taught an average of 1 2 percent of time. Our further analyses 
of the content matrix data (not shown) indicate that the specific topics reported most 
often under Number Sense were fractions, decimals, percent, and ratio and 
proportions, and the highest topics in Algebra were use of variables and multi-step 
equations. 

• Teaching of Algebra and Geometry is highly varied among teachers/classes. Time 
reported on teaching Algebraic Concepts varied from 1 8 to 40 percent of time across 
classes in both Initiative and Comparison schools. Time on Geometry varied from 6 
to 22 percent across classes in both groups of schools. 

• Expectations for learning as reported by teachers focused heavily on Understand 
Concepts and Perform Procedures (over 20% each). Expectations by teachers that 
students Analyze/ Reason and Memorize were slightly less ( 1 5% each), and Integrate 
Concepts and Solve Novel Problems were expected the least time. Understand 
Concepts refers to students’ ability to represent a concept, apply it to a problem, or 
explain its use. Perform Procedures means using numbers for counting or ordering, 
doing computation, or solving equations. Solve Novel Problems means doing non- 
routine problems or those for which the student has no routine strategy or algorithm. 

• Middle level teachers in Initiative schools did not report significantly different 
distributions of time than did Comparison school teachers on any of the five broad 
subject areas. One can observe small differences in the topic/expectations 
intersections. Teachers in Initiative focused slightly more on Solve Novel problems 
and Integrate when teaching Number sense than did Comparison teachers. Teachers 
in Comparison schools focused slightly more time on Analyze/Reason when teaching 
Algebraic concepts, but also had wider variation in expectation for Analyze/Reason. 
Initiative teachers placed slightly more time on the expectation of Integrating 
concepts. 
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Chart 3 

Elementary School Science Content Map 
Cross-State Sample 
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Chart 4 

Elementary School Science Content Graphs 
Cross>State Sample 
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Chart 5 

Middle School Mathematics Content Map 
Cross-State Sample 
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Chart 6 

Middle School Mathematics Content Graphs 
Cross-State Sample 
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Multiple Assessment Strategies in Math and Science 

Mathematics and science teachers are being encouraged to make use of a variety of assessment 
strategies, rather than relying on a single type of assessment, such as paper-and-pencil tests comprised 
of objective items or routine procedural problems. The purpose of varied assessment strategies is, in 
part, to increase the validity of the inferences that teachers can make about student learning. Using 
multiple sources of evidence allows the strengths in one type of assessment to compensate for 
weaknesses in another. But to what extent are mathematics and science teachers moving beyond a 
reliance on a single type of assessment, and what other strategies are they using? The results in Chart 
7 illustrate how Survey data can be used to find out. 

The Summary Scale on Multiple Assessment Strategies in science is based on several Survey items. 
Together these items provide a reliable index of the degree to which science teachers are using 
multiple assessment strategies in the classroom. (The average teacher scale score is set at 0 using a 
statistical procedure.) 

• The results show wide variation, with the greatest variation amongst middle school 
teachers from Initiative schools. That is, some of the middle school science teachers 
in Initiative schools appear to use greater variety in their assessment strategies than 
their counterparts in the Comparison schools. 

Science Item Profiles. The responses from science teachers on use of a variety of assessment strategies 
are displayed in the middle section of the chart and reveal useful results: 

• Portfolios. First, we note significant differences in the use of the reported strategies 
in science and wide variation in the use of portfolios . Some teachers are not using 
portfolios at all, while some middle school science teachers are using portfolios in 
assessment 1 to 3 times per month. 

• P erformance tasks are used to a significantly greater degree by Initiative teachers than 
Comparison teachers at the elementary and the middle grade levels. 

• Extended response. T eachers report that students are asked to write explanations for 
answers, on average, about 1 to 3 times per month. 

Mathematics Item Profiles. The responses from math teachers on several of the assessment questions 
are displayed in the third section of the chart: 

• Short answer. The most common assessment of student knowledge in class is short 
answer questions, such as asking students to perform a mathematical procedure. 
Students are given this type of assessment, on average, once a week. This is evident 
across grade levels and with both Initiative and Comparison schools. 

• Extended response. Middle grades teachers are significantly more likely than 
elementary teachers to ask students to explain or justify their answers (1-3 
times/month). There is relatively little use of objective items for mathematics 
assessment, especially in elementary classrooms where the average is a few times per 
year. 
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Chart 7 

Multiple Assessment Strategies in Math and Science 
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Use of Educational Technology and Equipment 

An important indicator of active, inquiry-based methods of teaching science and math, as well as 
school system capacity for supporting this approach, is the availability and use of educational 
technology and laboratory equipment. Science and mathematics standards advocate learning to apply 
knowledge to real problems and to gain skills that will be used outside of school. Science and 
mathematics applications in careers now involve computers, calculators, and a variety of simple and 
complex lab equipment. Thus, a key component of the Survey of Enacted Curriculum concerns the 
use of equipment and technology in teaching science and math. Chart 8 illustrates how the data can 
be reported to examine several kinds of questions concerning availability and instructional uses. 

Use of Technology. The Summary Scale at the top of Chart 8 includes six items that ask 
teachers to report the extent of calculator and computer use in class and how they are applied in 
instruction. The scale shows wide variation among math classrooms. Elementary grades use of 
technology is much more varied than middle grades. At both grade levels, Comparison schools have 
slightly lower scores on the summary scale and more variation among classrooms. 

• Calculator use. Teacher reports on student use of calculators indicate that middle 
grades consistently use them at least weekly, while elementary classes vary from less 
than monthly to weekly. Graphing calculator use varies widely: about one-third of 
middle grades classes do not have them, and the average class rarely uses them. 
Initiative school classes use graphing calculators more often than Comparison classes. 

• Instructional applications. The Survey data show that educational technology's most 
frequent uses in math instruction are Learning Facts or Practicing Procedures 
(average 25%) and Displaying/ Analyzing Data ( 1 8%). In science classes, calculators 
and computers are often used for Retrieving Information or Displaying/ Analyzing 
Data (both 18% avg.). However, these uses vary widely, from 0% o 40% of time, 
showing that teachers make very different or no use of educational technology. 

Equipment in Science. We highlight two uses of science equipment in classrooms: 

• Running water (a traditional indicator of lab capacity) is not available in about one- 
third of elementary classes and rarely used in the average elementary class. The 
average middle grade science class uses running water less than monthly, although 
classes in Initiative schools average monthly to weekly use. 

• Recently, "high-tech" approaches to experimentation in classrooms involve computer- 
lab interfacing devices, often called computer-based labs (CBLs). The sample data 
in the study indicate that one-third of classes did not have CBLs available, while the 
average class had access but rarely used them. About one-third of middle grades 
classes used them several times per year. 
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Chart 8 

Use of Educational Technology and Equipment 
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Influences on Curriculum and Practices 

The Survey includes questions for teachers aimed at the main influences on their classroom curriculum. 
In some schools, textbooks and materials that are selected by districts, states, or schools may be a 
major influence on what is taught. Many states and districts have established standards for student 
learning in mathematics and science, and these standards have been guided or influenced by national 
standards developed and published by professional organizations (NCTM, AAAS, NRC). Even where 
widely disseminated and used by teachers, standards do not provide curricula for teaching in 
classrooms . T eachers may rely on their own knowledge and experience, their colleagues, or mandated 
assessment programs to determine what is taught. The question of influences on curriculum is 
important for analyzing commonalities in curriculum and for determining how change and 
improvements can be made in science and math education. 

Chart 9 provides Cross-State Sample results for mathematics and science as reported by teachers in 
the 1999 Survey concerning major influences on their curriculum and teaching practices. 

• Influences on Mathematics. The item profiles (in the middle section) indicate that the 
most consistent positive influences reported by teachers were state 
frameworks/standards and textbooks/materials. State tests and national standards had 
slightly less influence on average, and there was greater response variation concerning 
state tests, with a significant portion reporting a negative influence. Teachers in 
Initiative schools had a more positive response regarding influence of state tests, and 
Initiative teachers had a significantly more positive response to national mathematics 
standards. 

• Influences on Science. The item profiles indicate that state frameworks/standards and 
textbooks/materials were reported by teachers as consistently positive influences, with 
state tests and national standards slightly less important. At the middle grades level, 
state science frameworks and state tests were a greater influence on teacher 
curriculum in Comparison schools than in Initiative schools. 

• Summary Scale results for influences on mathematics indicate wide variation in the 
influence of state and national math education standards on classroom curriculum. 
Middle school mathematics is influenced slightly less than elementary curriculum by 
standards, and elementary curriculum in Initiative schools receives a slightly stronger 
positive influence from standards than curriculum in Comparison schools. 
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Chart 9 
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Alignment of Content Taught with State Assessments 

The content matrix portion of the Survey of Enacted Curriculum provides a useful analysis function 
beyond simply reporting curriculum content taught. Eight states participating in the Study of State 
Initiatives volunteered to conduct an analysis of their state assessments in mathematics and science in 
relation to the content matrix. The results of the assessment analysis provide data for conducting 
correlational analyses of the teacher-reported class content and the content covered by the state 
assessment for the same grade and subject. (See the Reference Guide for a sample page from the 
content matrix section of the Survey.) 

The assessment analysis was conducted by teams of experts, generally 4 to 5 people per assessment 
subject/grade. The experts included state math and science specialists and university-based 
mathematicians, scientists and educators of these subjects. The teams to conduct the analyses were 
chosen by CCSSO and WCER. The analysis teams were trained in coding procedures and developed 
a common set of rules and criteria forjudging items for placement in the content matrix by topic and 
expectation. The same criteria and procedures were used in an analysis and coding of items from the 
1996 National Assessment of Educational Progress (NAEP) in Mathematics and Science. 

Mathematics Alignment. Chart 10 shows the alignment analysis for one state (not identified) 
in grade 4 mathematics, including the teacher-reported data on content taught, the analysis of state 
assessment items, and the same analysis of NAEP assessment items for grade 4. 

• The state assessment and NAEP assessment focused strongly on Operations, Number 
Sense, and Performing Procedures. Teacher reports of content taught show greater 
distribution to content topics and expectations than the coding of the test items. 
Teachers indicated that some time was spent on all six expectations across Number 
Sense, Operations, Measurement and Geometric Concepts. The alignment statistic 
of .37 means that less than half the intersections of content topics by expectations 
reported by teachers were in common with the assessment items found on the state 
mathematics test and on the NAEP test. 

Science Alignment. Chart 1 1 shows the alignment analysis for one state (not identified) in 
grade 8 science, including the teacher-reported data on content taught, the analysis of state assessment 
items, and the same analysis of NAEP assessment items for grade 8. 

• The NAEP assessment focused strongly on Life Science, Physical Science, 
Understand Concepts and Perform Procedures, and some emphasis on Analyze 
Information. The state assessment focused on Physical, Earth, and Life Sciences, 
and on Memorize and Understand Concepts. Teacher reports of content taught 
indicate greater distribution to the six expectations in the content areas of 
Physical, Life, and Earth Sciences than the coding of the test items. The 
alignment statistic of .33 means that less than half the intersections of content 
topics by expectations reported by teachers were in common with the assessment 
items found on the state science test. 
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Chart 10 
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Chart 11 
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Teacher Preparation 

A key feature of the Survey of Enacted Curriculum is the section requesting information on formal 
preparation, including college major and number of college courses related to teaching mathematics or 
science. Questions also ask about the amount of professional development and training activities 
received by teachers in the past year, teacher reactions, and use of their development. Finally, the 
Survey asks for teacher views on how well prepared they consider themselves for using a variety of 
teaching strategies and for teaching with various groups of students. The Survey results for the Cross- 
State Sample are shown in Charts 12 and 13. 

Mathematics 

• Professional Development. Middle grade teachers received an average of 6 to 15 hours of 
professional development in 1 2 months on in-depth study of math content and the same amount 
of time on methods of teaching math. Elementary teachers averaged less than 6 hours in each 
area. At both levels, there are significant differences in professional development time at both 
levels, varying from a total of less than 6 hours to a total of almost 40 hours. Teachers in 
Initiative schools received more professional development in math than teachers in Comparison 
schools. (The Survey includes questions on teachers’ reactions to professional development, 
and the data indicate areas most used by teachers are: new curriculum, standards, teaching 
methods, multiple assessment strategies, and needs of all students). 

• Teacher Major. Almost half of the middle grades math teachers reported having a major in 
math or math education compared with 5% of elementary teachers. Initiative schools reported 
about 10% more teachers with math majors. 

• Innovative Strategies. Teachers reported they are well prepared, on average, to use 
cooperative learning groups, to integrate math with other subjects, and to teach with 
manipulatives. With each of these innovative strategies, teacher responses were varied-a 
significant portion of teachers were only somewhat prepared, while many were very well 
prepared. 

Science 

• Professional Development. Middle grades science teachers received an average of nearly 20 
hours of professional development both in methods of teaching science and study of science 
content. Elementary teachers received about 1 0 hours less development in each area. Teachers 
in Initiative schools received significantly more science education development at both 
elementary and middle grades levels. (Data are available to analyze professional development 
activities by the reactions and use by teachers, and in science, the highest use is with activities 
in student assessment, in-depth content study, new teaching methods, and implementing state 
or national standards.) 

• Teacher Major. Almost 60% of middle grades science teachers have a major in science or 
science education compared with 10% of elementary teachers. In Initiative schools, 70% of 
middle grades teachers had a science major. 

• Innovative Strategies. Teachers reported they are well prepared, on average to integrate 
science with other subjects, to manage hands-on or lab activities, and to use a variety of 
assessment strategies. With each of these innovative strategies, teacher responses varied, 
particularly with elementary teachers. About one-third of elementary teachers reported that 
they are not well prepared to teach with any of these three strategies. 
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Chart 12 

Teacher Preparation - Mathematics 
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Chart 13 

Teacher Preparation - Science 
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Uses of Enacted Curriculum Data 



In the 1990s, states and school districts have worked to improve the quality of mathematics and 
science teaching through a variety of policy and program initiatives, including more challenging 
content standards, professional development on teaching methods and subject knowledge, new 
curriculum materials, and system wide student assessments. As initiatives have been developed and 
implemented, and goals for student achievement have been raised, educators and decision makers have 
sought information to evaluate progress of change in instruction and to assess the needs of teachers, 
particularly in instructional practices and delivery of curriculum that will meet challenging standards 
in mathematics and science. Reliable, comparable state-level indicators have been developed and 
reported (Blank, et al., 1999; Reese, et al., 1997); however, broad state-level indicators have typically 
not provided sufficient detail about the breadth and depth of curriculum content and practices actually 
taught in classrooms. 

The charts and description of sample results from the multi-state study, outlined above, highlight the 
kinds of findings and data possible with the Survey of Enacted Curriculum, and these examples 
illustrate how the data can be reported and analyzed. Curriculum data can serve a variety of 
audiences— teachers, curriculum specialists, administrators, parents, and policymakers. Used properly, 
the survey data can be extremely helpful to school systems. 

Survey Topics. The survey is designed to be completed through self-report by teachers. In the 
1999 study, a sample of teachers in a school were selected, but the data are likely to be even more 
useful when all teachers in a school or several grade levels across schools are surveyed. The total 
survey includes 155 questions plus a subject content matrix. The topics can be summarized by the 
following outline: 

■4 Subject Content 

• Curriculum topics taught during school year 

• Teacher expectations for student learning by topic 
^ Classroom Instructional Practices 

• Instructional activities in a subject— main types and specifics 

• Use of calculators, computers, lab equipment 

• Assessment strategies 

• Time allocation in recent unit 

• Homework 

• Class and student characteristics 
■4 Teacher Preparation 

• Professional development, time and topics 

• Course preparation of teachers in math/science 

• Influences on teacher decisions about curriculum and instruction 

• Teacher perceptions of areas of preparation 

• Teacher views of school conditions for teaching 
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Applications of the Data. Reliable, comparable data on curriculum and teaching as they are 
actually delivered in classrooms are generally not available to teachers, specialists, or administrators. 
In the past, the methods of collecting such data were either too costly or difficult to analyze and 
compare across classes, schools, and districts. Now, data from Surveys of Enacted Curriculum can 
be useful to several audiences. 

State and local decision-makers can use data on instructional practices and content to inform policy 
decisions, plan professional development, and evaluate the effectiveness of reform initiatives in 
changing practice. Principals and other administrators can use instructional activity measures to 
inform parents and others of practices within a school, district, or state, and data can be analyzed by 
teacher background and level of preparation. Teachers will find the data useful for curriculum 
planning, for reflection on their own practice, and as a basis for pursuing collaborative discussions with 
colleagues about teaching strategies specific to content. 

We can summarize four primary categories of applications for the Survey data as identified by state 
education leaders participating in the project: 

/ Interpreting Student Assessment Results — The Enacted Curriculum Surveys can be given 
in their entirety, including moderately detailed questions on instructional practices and content 
(totaling up to 1 .5 hours response time), or selected components of the Survey can be used. The 
Survey can be given simultaneously with student assessments to help analyze results and develop 
strategies for instructional improvement. The subject content matrix data reported by teachers 
can be analyzed with the student assessment results to determine strengths and weaknesses in 
curriculum and teaching strategies. 

S Aligning Curriculum with Standards — Almost every state and many districts have 
developed content standards for student learning. The Enacted Curriculum Surveys can provide 
a database for monitoring the degree to which classroom curriculum is moving toward the 
standards. Standards are written with specific benchmarks or indicators of student performance, 
and the Survey data are reported both by broad categories matched to standards and by specific 
item profiles and teacher expectations that match to the benchmarks. 

y Needs Assessment, Program Evaluation, Planning — State departments of education and 
local districts are responsible for evaluating programs for improving instruction or professional 
development. The Enacted Curriculum Surveys can provide a tool for identifying needs, 
determining effects on curriculum, or planning programs. 

/ School Curriculum Improvement — Teachers, schools and districts often seek ways to 
improve dialogue among teachers regarding their own practice and curriculum content. Working 
with Enacted Curriculum data can provide important incentives, as well as comparable measures 
for starting more in-depth discussion about how teachers can improve their own practice, 
advance articulation of curriculum between grades/courses, and share strategies for 
improvement. 



0 
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Department heads, curriculum specialists, professional development leaders, and administrators and 
teachers may want to use the enacted curriculum data as a basis for working together in analyzing 
differences and similarities in curriculum and teaching practices. They also may want to use these 
basic indicators to begin more detailed discussion of the range of teaching strategies used in classes, 
and to plan how improvement might be made in math and science teaching. 



Survey Development and Procedures 

The Survey of Enacted Curriculum was originated by CCSSO as a component of the Science 
Education Project of the State Collaboratives on Assessment and Student Standards 
(SCASS)(CCSSO, 1998; Martin, Blank & Smithson, 1996). A grant from the National Science 
Foundation provided support for CCSSO and states to develop, demonstrate, and test survey 
instruments for classroom curriculum in science and mathematics and methods of analyzing and 
reporting data. A multi-state team of educators, assessment specialists, and researchers worked 
together to design and edit the surveys. Major concepts for development of surveys came from state 
and national standards for student learning and teaching. Example items and response formats were 
adapted from national surveys (NCES/NAEP, 1 997; Weiss, 1 994) and the TIMSS international survey 
(NCES, 1996). 

Survey Design. Prior work by Andrew Porter and colleagues at the Wisconsin Center for 
Education Research in the “Reform Up Close” study (1993) and other research on curriculum change 
(1998) had demonstrated the validity of teacher self-report surveys as compared to results obtained 
through daily logs of content and instruction, interviews, and observation. Porter and John Smithson 
from the Wisconsin Center provided research expertise and data collection and analysis leadership for 
the 1999 State Initiatives study. 



The Survey instruments in science and mathematics were complected prior to the 1 998-99 school year. 
The surveys for this study were designed for elementary teachers, middle level teachers and high 
school teachers. One main goal of the Survey is to analyze classroom practices across grade levels, 
and many of the Survey sections and items are identical between grade levels. The major section that 
differs is the content matrix in math and science, which has more content topics in the higher grade 
levels. 

The designed instruments were pilot-tested in summer 1998, and the teachers participated in a focus 
group discussion aimed toward survey improvements. In the Spring 1999 data collection, one-fourth 
of teachers participating in the Survey were asked to conduct brief surveys with one class of their 
students. These items focused on classroom practices and activities for which students would be most 
able to report. The data were intended for validation analyses of the teacher responses. 

Sampling and Data Collection. State superintendents of education were contacted in Fall 1998 
to determine interests in states participating in a Study of State Initiatives in Mathematics and Science 
Education. Superintendents and their staff were provided information on the study objectives and the 
intended survey approach to data collection and analysis. The study was designed to use a common 
data collection approach, but the state initiatives used as the basis for sample selection could vary. 
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Each state was asked to provide funding for travel expenses for two staff to participate in collaborative 
team meetings. 

Eleven states agreed to participate: Iowa, Kentucky, Louisiana, Massachusetts, Minnesota, Missouri, 
North Carolina, Ohio, Pennsylvania, South Carolina, West Virginia. Six of the 1 1 states had been a 
part of the State Systemic Initiatives (SSI) program supported by the National Science Foundation. 
These states typically had multiple program and policy initiatives including standards, curriculum 
materials, professional development, and assessment. In several states the initiatives focused on 
assessment development and training for teachers. Several states also focused on implementing new 
content standards for science and math. 

State leaders were asked to select schools and teachers to participate in the study based on their state 
initiative. The initiatives varied by state but methods of sampling were consistent. Each state was 
asked to include schools from urban, suburban, and rural districts, and schools were to vary in size and 
student composition. Each state was given the following sampling plan: 

Goal: Total of 160 teachers and up to 80 schools per state 

• 20 schools at each of two grade levels (e.g., elementary, middle) 

10 “Initiative” schools and 10 “Comparison” schools (matched on 
enrollment, poverty level, community) 

• Per school: 2 teachers of science, 2 teachers of math 

Schools added if 4 teachers were not present per school. 
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Analysis Guide 



The following pages of this report provide explanatory materials, including: 

• Sample sections of the Survey of Enacted Curriculum; 

• A guide for Interpreting Content Maps shown in the report; 

• Charts on Use of Class Time; 

• Descriptive Data about the surveyed schools, teachers, and classes; 

• Information on State Initiatives, Standards and Assessments. 

Teacher Reporting on Subject Content. The Subject Content section of the Survey requests 
information regarding topic coverage and teacher’s expectations for students in the target mathematics 
class for the current year. It is not intended to reflect any recommended or prescribed content for the 
grade level. For Middle School Mathematics, six content areas are surveyed: Number 
Sense/Properties/Relationships, Measurement, Data Analysis/Probability/Statistics, Geometric 
Concepts, Algebraic Concepts, and Instructional Technology. (The figure on the following page shows 
one area, Number Sense.) 

To complete this section, the teacher identifies topic/sub-topic areas covered in his/her mathematics 
class, using “time on topic” column (0=none - not covered, l=slight coverage - less than one 
class/lesson, 2=moderate coverage - one to five classes/lessons, 3=sustained coverage - more than five 
classes/lessons). 

Then the teacher indicates the relative emphases of each Expectation for Students for every sub-topic 
taught using scale bubbles for six categories: Memorize, Understand concepts, Perform procedures, 
Analyze/reason, Solve novel problem, and Integrate. Four scale bubbles indicate: 0=no emphasis, 
l=slight emphasis - accounts for less than 25% of the time, 2=moderate emphasis - accounts for 25% - 
33% of the time, 3=sustained emphasis - accounts for more than 33% of the time. 

Teacher reporting on Instructional Activities. See example section from Mathematics survey 
on following page. 
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Sample Sections from Survey 



Subject Content : Mathematics 



Time on Middle School 

Topic Mathematics Topics 



Expectations for Students in Mathematics 



<none> 
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Understand 

Concepts 
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Procedures 



Analyze/ 
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Problems 
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INSTRUCTIONAL ACTIVITIES IN MATHEMATICS 

Listed below are some questions about what students in the target class do in mathematics . For each activity , pick one of 
the choices (0, 1 , 2, 3) to indicate the percentage of instructional time that students are doing each activity. Please think of 



What percentage of mathematics instructional time in the target class do students: 



NOTE: No more than two ‘3's , or four ‘2's should be reported for this set of items. 






None 


Less than 


25% to 33% More than 33% 
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Watch the teacher demonstrate how to do a procedure or 
solve a problem. 
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Read about mathematics in books, magazines, or 
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Collect or analyze data. 
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Maintain and reflect on a mathematics portfolio of their 
own work. 
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Use hands-on materials or manipulatives (e.g., counting 


( 5 ) 
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blocks, geometric shapes, algebraic tiles). 
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Engage in mathematical problem solving (e.g., 
computation, story-problems, mathematical 
investigations). 
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Take notes. 
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Work in pairs or small groups (non-laboratory). 
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Do a mathematics activity with the class outside the 
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Use computers, calculators, or other technology to learn 
mathematics. 
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Work individually on assignments. 
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Take a quiz or test. 
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Interpreting Content Maps 

Content maps provide a three-dimensional representation of instructional content using a surface area chart which 
results in a graphic very similar to topographical maps. The grid overlaying each map identifies a list of topic areas 
(indicated by horizontal grid lines; see O below) and six categories of cognitive expectations for students (indicated by 
vertical lines; see © below). The intersection of each topic area and category of cognitive expectation represents a 
measurement node (see © below). Each measurement node indicates a measure of instructional time for a given topic 
area and category of cognitive expectation based upon teacher reports. The resulting map is based upon the values at 
each of these measurement nodes. It should be noted that the spaces between each measurement node, that is the 
surface of the map, are abstractions and are not based upon real data the image of the map is simply a computer 
generated graphic based upon the values for each intersecting measurement node. The map display is utilized to 
portray the third dimension (percent of instructional time; see © below) onto this grid utilizing shading and contour 
lines to indicate the percent of instructional time spent (on average across teachers) for each topic by cognitive 
expectation intersection. 

The increase (or decrease) in instructional time represented by each shaded band is referred to as the measurement 
interval (see © below). To determine the amount of instructional time for a given measurement node, count the 
number of contour lines between the nearest border and the node and multiply by the measurement interval. 



The graphic at left below displays the three dimensional counterpart of the image represented by the content map 
displayed on the right. Both graphs indicate that Understanding Concepts related to Number Sense and Operations 
occupies the majority of time spent on grade four mathematics instruction (9% or more of instructional time over the 
course of a school year). 




Source: CCSSO/WCER, Survey of 
Enacted Curriculum, 1999 



Measurement Interval = 1% 
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BEST COPY AVAILABLE 




Use of Class Time 

During Most Recent Unit of Instruction in Mathematics 



Legend 



-1 StD 



Mean 


Cross - State Sample: Mathematics 


By Grade Level 


Elementary 


Middle Sch. 


lillJlBIgB 

+ 1 StD 


El Middle (136) 

□ Elementary (189) 


ED Initiative (74) 

SO Comparison (66) 


El Initiative (63) 

E3D Comparison (52) 



How many class periods did 
your most recent math 
instructional unit cover? 



1 I ? I \ 

0 1 2 3 4 5 



0 1 



1 1 1 1 

2 3 4 5 



"1 I 1 1 1 

1 2 3 4 5 



Response Codes: 



0= 1-2 1=3-5 2 = 6-10 3= 11-15 

4 = 16-20 5 = 21 or more 



What percent of mathematics instructional time in this unit was spent on the following: 
(NOTE: Responses should sum to 100.) 



Admin, routines, interruptions, 
non-instructional activities. 

Whole class lecture or class 
discussion. 

Individual student work (e.g., 
complete exercises, read 
textbook). 

Small group work. 

Field-study or out-of-class 
investigation. 

Student demonstrations or 
presentations. 

Review or work on 
homework. 

.Test or quiz. 

Bring students up-to-date 
due to absences/transfers. 
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Source: CCSSO/WCER, Survey of Enacted 
Curriculum, 1999 



Indicates statistically 
significant mean 
differences. 
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Use of Class Time 

During Most Recent Unit of Instruction in Science 
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Cross - State Sample: Science 


By Grade Level 
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Response Codes: 


0 = 1-2 


1=3-5 2 = 6-10 


3 = 11-15 






4 = 16-20 


5 = 21 or more 







What percent of science instructional time in this unit was spent on the following: 
(NOTE: Responses should sum to 100.) 



op. 



w- j 



Admin, routines, interruptions 
non-instructional activities. 

Whole class lecture or class 
discussion. 

Individual student work (e.g., 
complete exercises, read 
textbook). 

Small group work. 



Student work with hands-on 
or laboratory materials. 

Field-study or out-of-class 
investigation. 

Review or work on 
homework. 

Test or quiz. 



Bring students up-to-date — PH' 
due to absences/transfers. 
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Source: CCSSO/WCER, Survey of Enacted 
Curriculum, 1999 



Indicates statistically 
significant mean 
differences. 
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BEST COPY AVAILABLE 



Descriptive Data on Schools, Teachers, and Classes Participating in the 
1999 Survey of Enacted Curriculum 



| Number of States 




11 


i 




Teachers Responding to Survey 








(whole or in part) 


Teachers 


Mathematics 


Science 




Elementary 


189 


151 




Middle 


136 


104 




High 


32 


17 



Schools by Enrollment 

(selected as Initiative vs. Comparison) 







Schools with Math or Science Survevs 






Initiative 


Comparison 


Not reported 




Under 300 


11 


10 


3 




301 - 600 


49 


38 


20 




601 - 1000 


42 


38 


7 




1001 or more 


19 


8 


2 




Not reported 


28 


22 


30 




Total 


149 


116 


62 




School % Poverty 














Schools with Math or Science Survevs 






Initiative 


Comparison 


Not reported 




0 - 34% 


77 


46 


13 




35 - 74% 


62 


64 


12 




75 - 100% 


8 


1 


7 




Not reported 


2 


5 


30 




Total 


149 


116 


62 




Class Reported by Teacher 










(1 class per teacher) 






Math 


Science 






Grade 3 


16 


14 






4 


155 


126 






5 or 6 


4 


10 






7 


19 


20 






8 


111 


75 






9 or higher 


30 


10 



Teaching Time 

Elementary (hours/week) 




Math % 


Science % 




Less than 4 


4.8 


44.9 




4-4.9 


16.2 


31.5 




5 or more 


79.1 


23.5 
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Teaching Time (continued) 
Middle Grades (hours/week) 




Math % 


Science % 


Less than 4 


20.2 


13.4 




4-4.9 


31 


34 




5 or more 


49 


52.6 



Achievement Level of Students 












Math % 


Science % 




High 


16 


9.7 




Average 


50.2 


47.2 




Low 


13 


14.1 




Mixed 


19.6 


29 



Teacher Characteristics 








Experience: Yrs. in Subject 




Math % 


Science % 


0-2 


12.8 


11.8 




3-5 


18.6 


13.7 




6-11 


21 


22.6 




12 or more 


47.6 


51.9 


Major: Bachelors or Highest 




Math % 


Science % 


Elementary Ed. 


40.6 


43.5 




Middle Ed. 


6.7 


4.2 




Math Ed. 


13.3 






Science Ed. 




12.6 




Mathematics 


10.5 






Science field 




11 




Other 


28.9 


26.7 






Math % 


Science % 




BA/BS 


51.4 


42.4 




MA/MS or higher 


48.7 


57.6 



Teacher Professional Development 

Content study in field (hrs. in last year) 




Math % 


Science % 


< 6 


32.6 


52.8 




6-15 


25.2 


21.6 




16 or more 


22.2 


25.7 


Methods of teaching in field 
(hrs. in last year) 




Math % 


Science % 


< 6 


47 


34.9 




6-15 


29.4 


46.7 




16 or more 


23.7 


18.4 



Teacher Demographics 












Math % 


Science % 




Female 


82.1 


76.8 




Male 


' 17.9 


23.2 




White 


93.9 


90.7 




Minority 


6.1 


9.4 
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Information on State Initiatives, Standards and Assessments 

The study of state reform is based on a design for surveying a selected sample of teachers and 
analyzing the data to determine effects of a state initiative in mathematics or science education on 
teaching practices and curriculum. In some states the initiative is directly linked to state content 
standards. In others, the initiative relates to a broader set of state education policies to improve 
education. Six of the 1 1 states were in the state systemic initiative (SSI) program supported by the 
National Science Foundation. Following is a summary outline of key state information upon which 
the study is based. 



Iowa 

Surveys, Spring 1999 Grades 

Mathematics 4, 8 

Science 4, 8 




State Initiative 


Year Implemented 


Mathematics: 


First Governor's Conference on Reform in Math Ed. (K-12) 
Science : 


1992 


New Standards project 


1992 


Science SCASS Assessment project 


1993 


Nat. Science Ed./NCTM Standards Awareness / Implementation 


1996 


State Content Standards 




(Standards and Frameworks developed at the District level) 




State Assessments 




(No statewide assessment) 





0 
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Kentucky 

Surveys , Spring 1999 Grades 

Mathematics 4, 8 

Science 4, 8 




State Initiative 


Year Implemented 


Appalachian Rural Systemic Initiative (ARSI) 
KERA — State Reform 


1990 


Partnerships for Reform Initiatives in Science and Mathematics (PRISM)-NSF/SSI 


1991 


Kentucky Middle Grades Mathematics Teacher Network 


4 years 


Eisenhower Regional consortium for Mathematics and Science Education at AEL 
Informal Science Organization/School Partnerships 


1993 


K - 4 Mathematics Specialist Program 


3 years 


State Content Standards 




Transformations: KY Curriculum Framework 


1995 


KY Core Content for Math and Science Assessment 


1996 


Program of Studies for KY Schools, PK-12 


1997 


State Assessments 




KIRIS Math/Science Gr. 4, 8 

Commonwealth Accountability Testing System (CATS) 


1991 


Math Gr. 5, 8; Science Gr. 4, 7 


1998 



Louisiana 

Surveys , Spring 1999 Grades 

Mathematics 4, 8 

Science 4, 8 

State Initiative 

Math & Science: LA Systemic Initiatives Program 
K-3 Reading and Mathematics Initiative 
Developing Education Excellence and Proficiency 
Math:Gr. 3-8, Science: Gr. K-12 

State Content Standards 

LA Mathematics and Science Content Standards (1997) 

LA Mathematics and Science Curriculum Frameworks (1995) 

State Assessments 

Math: CRT Gr. 4,8 NRT Gr. 3, 5, 6, 7, 9 

Science: CRT Gr. 4,8 NRT Gr. 3, 5, 6, 7, 9 
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Massachusetts 

Surveys, Spring 1999 Grades 

Mathematics 4, 8 

Science 4, 8 

State Initiative Year Implemented 

Partnerships Advancing the Learning of Math and Science (PALMS): 1 992 

PK-12/ Higher Education Goals: — Increase student achievement in math, 

science and technology. Reduce achievement gaps for ethnic, bilingual, and gender groups 

Focal Areas: 

1) Data-driven systems 

2) Standards-based curriculum, instruction, assessment 

3) Qualified/quality teachers 

4) Middle school/high school and transitions 

5) Parent/community involvement 

State Content Standards 

Mathematics Curriculum Framework: Achieving Mathematical Power (1995) 

Science and Technology Curriculum Framework: Owning the Questions 
through Science and Technology Education (1995) 

State Assessments 

MCAS (began *97-98): Math, Science: Grades 4, 8, 10 



Minnesota 

Surveys, Spring 1999 Grades 

Mathematics 4, 8 

Science 4, 8 

State Initiative Year Implemented 

Minnesota Graduation Standards Fall 1998 

Basic standards (R, W, Math) and High Standards in 10 areas including Math 
and Science. 

Math reform schools: Have implemented NSF math curricula. Non-reform: 
traditional, textbook programs. 

Science reform schools: Some, not all using kit-based programs-FOSS or STC; Others, with teachers in the 
"best practice network in science" 

State Content Standards 

Mathematics K-12 Curriculum Framework (1997) 

Science K-12 Curriculum Framework (1997) 

State Assessments 

Basic Stands. Test Math, Gr. 8 (3 yrs.) 

Comp. Assess. Gr. 3,5 (began 97-98) . 
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Missouri 

Surveys , Spring 1999 Grades 

Mathematics 4, 8 

Science 3, 7 

State Initiative 

“Initiative” schools were selected from two programs: 

(1) Schools that participated voluntarily in the first year that state-level performance-based math and science 
assessments became available. Mathematics assessment began Spring 1997, and implemented statewide 1998. 
Science assessment began Spring 1998. 

(2) Schools in districts that participated in the voluntary inservice training program on performance-based 
assessment, which began being offered by the state in 1993. 

State Content Standards 

Math Curriculum Frameworks (1996) 

Science Curriculum Frameworks (1996) 

Show Me Standards (1996) 

State Assessments 

Missouri Assessment Program 

Math: Grades 4, 8, 10 (began 1996-97) 

Science: Gr. 3, 7 (began 1997-98) 



North Carolina 

Surveys, Spring 1999 Grades 

Mathematics 4, 8 

State Initiative 

"The ABCs of Public Education" is an initiative that began in 1992. There are three parts: A: Accountability; 

B: Basics and High Education Standards; and, C: Maximum local control. Key aspects of each part: 

A — Individual schools held accountable, staff responsible, students tested in grades 3-8, high school end 
of course tests, and schools judged on raising achievement. 

B — State standards in Reading, Writing, and Mathematics; and grade specific objectives per content 
area and tests based on objectives. 

C — Principals and teachers make decisions on materials and instruction; state provides information on 
"best practices,” curriculum standards, and technology. 

State Content Standards 

NC Standard Course of Study, Mathematics Competency-based Curriculum Teacher Handbook K-12 (1994) 
Strategies for Instruction in Mathematics (state-provided for each grade) 

State Assessments 

North Carolina Testing Program: Math: Gr. 3-8 (began 1992), Algebra 1 (began 1989) 



O 
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Ohio 

Surveys, Spring 1999 


Grades 


Mathematics 


4,8 


Science 


4,8 


State Initiative 


Year Implemented 


Urban Schools Initiative 


1996 


The Urban Schools Initiative was 


launched by the Ohio Department of Education to comprehensively address the 


challenges facing urban school communities. The initiative represents all twenty-one of Ohio's urban school 
districts, 24% of the states total student population, and 72% of its minority students. The Urban Schools 
Initiative (USI) has been a leader in developing and implementing new programs, attracting grants and making a 
positive impact on students. With its District Team, School Readiness Resource Centers, Professional 
Development and Disciplinary Intervention Grants and its widely circulated report, Through the Eyes of Children, 
Ohio's USI has had a substantial impact on the state's urban school communities. 


State Content Standards 




Model Competency-Based Mathematics Program (1990) 
Model Competency-Based Science Program (1994) 


State Assessments 




Ohio Proficiency Test Program 
Mathematics: 4, 6, 9, 12 
Science: 4, 6, 9, 12 






Pennsylvania 

Surveys , Spring 1999 


Grades 


Mathematics 


4,8 


Science 


4,8 


State Initiative 




State Content Standards 




Mathematics Curriculum Framework (1996) 
Science and Technology Framework (1995) 


Mathematics Standards (1999) 




State Assessments 




Math: NRT Gr. 5, 8, 1 1 
Science: Developing 
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South Carolina 

Survey Grades 

Science, Spring 1999 4, 8 

Mathematics, Fall 1999 4, 8 




State Initiative 


Year Implemented 


SC State Systemic Initiative (K-12) - reform, standards implementation 1992 

Teachers Leading Teachers (Gr. 4-8) - reform, physical science content 


Science SCASS Assessment project (K-12) 

Nat. Sci. Standards- Building a Resource (K-12) 


1995 


Instructional Improvement Initiative (K-12) — Low performing schools, 


to increase student achievement 




State Content Standards 


State Assessments 


SC Science Framework (1996) 


Basic Skills Assess. 


SC Acad. Achievement. Standards for Science (1996) 


Science: Gr. 3, 6, 8 


SC Science Curricul. Standards (1998) 

Expert review and Revisions to Sci. Standards (1999) 
Mathematics Standards (1998) 

Mathematics Framework (1993) 


Math: Gr. 3,8,10 



West Virginia 

Surveys , Spring 1999 Grades 

Mathematics 8, 10 

Science 8, 10 






State Initiative 




Year Implemented 


Mathematics Reform: Project MERIT 
(Mathematics Education Reform Initiative for Teachers) 

Focus on the way mathematics is taught particularly in grades 6-10. 
Schools were selected for SEC based on their participation in the project. 


1998 


Science Reform: Project CATS 1995 

(Coordinated and Thematic Science) 

Focus on integration of science curriculum particularly grades 6-10. 

Initiative schools were selected for SEC based on the schools with teachers trained in CATS. 


State Content Standards 


State Assessments 


Instructional Goals and Objectives for Mathematics (1996) 
Instructional Goals and Objectives for Science (1996) 


NRT: 


Math, Gr. K-ll; 
Science: Gr. 3-11 
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